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ABSTRACT 

This paper discusses the follow-up assessment of 
University of Kentucky teacher education graduates. An instrument was 
developed and administered to first-year teachers and their 
internship committee members (resource teachers, principals, and 
teacher educators) participating in the Kentucky Beginning Teacher 
Internship Program- Respondents were asked to identify the degree of 
difficulty that the intern teachers experienced in each of 20 
-competency areas. Over 4 academic years, approximately 70 percent of 
the teacher education graduates have responded to the survey and 
given permission for committee members to participate. Results are 
passed on to the various college program faculties, who use the 
findings to create new courses, revise existing courses, and develop 
in-ervice workshops. Overall, both quantitative and qualitative 
responses have been complimentary of the University's teacher 
education programs. Comments from graduates have consistently 
suggested that field experiences be increased, courses and 
instructors within the professional sequence be better linked, theory 
be directly related to practice, and instruction of political and 
administrative aspects of teaching be expanded. Appendices provide: 
(1) a table of the percentage of graduates and their internship 
committee members rating graduates as proficient or above on selected 
teacher competencies, and (2) a copy of the assessment instrument. 
(JDD) 
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Determining the Success of Teacher Preparation by Assessing What Teacher 
Education Graduates Know and Are Able To Do 

Introduction 

Educational reports, the media, and the teaching profession itself have recently 
focused somewhat less than favorable attention on the performance of new teachers and 
the quality of the teacher education programs that prepare these teachers. These images 
of new teachers portray them as ill-prepared for the duties and responsibilities of the 
classroom. In response to this attention, it is imperative that teacher education 
institutions continuously monitor and assess the effectiveness of their programming. 

In its redesigned accreditation standards, the National Council for Accreditation 
of Teacher Education (NCATE) advocates the importance of follow-up studies of 
teacher education graduates. The compliance criteria associated with standard II. B, 
"Relationships with Graduates," states: 

(1) The unit keeps abreast of emerging evaluation techniques 
and engages in regular and systematic evaluations, 
including follow-up studies, to determine the success and 
quality of graduates in the professional education roles for 
which they were prepared. 

(2) The results of evaluation efforts, including follow-up 
studies of graduates, are used by the unit to modify and 
improve programs. (NCATE, 1992, p. 52) 

Although teacher preparation institutions generally agree that follow-up studies 
would provide valuable feedback for directing program development and improvement 
strategies, and although NCATE and some state departments of education require 
follow-up studies, many institutions continue to experience problems in designing and 
implementing effective teacher and employer follow-up procedures. Evidence to 
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support this statement can be found as one examines the number of teacher education 
institutions that are having difficulty meeting the NCATE standard related to 
relationships with graduates, specifically the criteria related to follow-up studies. 

Typically, follow up studies have consisted of mailed surveys containing 
questions relating to graduates 1 levels of satisfaction with their programs of study. 
Graduates are asked to identify strengths and weaknesses of their programs and to 
provide information about their intentions for further study. The information provided 
is often so general that its usefulness is insignificant. In addition, the data collected 
from most such studies reflect only respondent self analysis of teaching effectiveness. 
Thus, no comparisons with the knowledge and performance of other new teachers can 
be made. 

For several years, the College of Education at the University of Kentucky 
utilized this traditional approach to learning more about its programs from its 
graduates. The survey instrument used consisted of general, open-ended questions 
relating to graduates' perceptions of their preparation program and their instructional 
effectiveness. The instrument was neither normed or criterion referenced. Responses 
were generally below twenty percent, and respondents tended to give non-specific 
answers to survey questions. Such data were not prescriptive to those faculty 
responsible for program quality and revision. Consequently, it became clear to faculty 
concerned with assessing program effectiveness that a more objective and exacting 
follow-up process was needed. This paper will describe the process used to develop the 
current assessment of University of Kentucky teacher education graduates, selected 
findings, and examples of how findings have been used for program development. 
Also presented are some revisions of the follow-up assessment currently being piloted 
to reflect the focus on outcomes-based education reform efforts in Kentucky. 



Development of the Assessment Process 

Encouraged by the efforts in Georgia in 1980 to implement a new teacher 
assessment system and the development and adoption of a summative, new-teacher 
observation instrument by Florida in 1983, Kentucky initiated the Kentucky Beginning 
Teacher Internship Program in 1985. This program is a support program for first-year 
teachers that also involves a formal evaluation component. Three team members- 
typically a resource teacher in the new teacher's content area, the new teacher's 
principal, and a teacher educator-formally observe the new teacher periodically 
throughout the year and meet with the new teacher to discuss the preparation and 
progress of the new teacher's professional development plan. 

University of Kentucky faculty were involved with this program at its inception. 
While involved, several became aware of the ways the internship team experience could 
enhance the curriculum revision process. Their opportunity to observe the classroom 
practices of graduates and to identify the strengths and weaknesses of such practices 
allowed them to return to their campuses with ideas for program improvement. Several 
of these participants soon began to explore ways that the University might be able to 
take advantage of the opportunities provided by the Kentucky Teacher Internship 
Program to improve the College's follow-up process, and Sandidge (1989) engaged in 
research to determine intern teachers' and resource teachers* perceptions of beginning 
teacher problems, concerns, and support within the context of the Kentucky Beginning 
Teacher Internship Program. 

Development of the Instrument 

In order to develop an instrument that could be used to identify competencies of 
first-year teachers, Sandidge (1989) examined a number of earlier studies of the 
problems typically encountered by novice teachers. Included was a study conducted by 
Shelley (1979) that examined the logs kept by teachers during their first three months 
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of teaching to determine the problems they experienced and the severity of the 
problems. Major problems included group and individual disruptions; inability of new 
teachers to plan instruction to match students' differing ability levels; communications 
with parents; difficulty in giving clear directions during classroom instruction; and 
routine management procedures such as lunch counts. 

An ethnographic study of first-year teachers conducted by Applegate, et. al. 
(1977) revealed that novice teachers expressed problems with managing the classroom, 
handling discipline, making and upholding decisions, evaluating students, and 
separating their personal and professional lives. They also indicated that they 
experienced problems with student opinions of the teacher, opinions and actions of 
colleagues, lack of parental concern, relationships with administrators, operating 
equipment, managing paperwork and time, and organizing curricula. 

Sweeney (1984) surveyed new teachers, experienced teachers, cooperating 
teachers, and principals regarding their perceptions of the problems of new teachers. 
New teachers were perceived as least adequate in understanding and managing behavior 
problems in the classroom, managing mainstreamed students with handicapping 
conditions, working with students with learning problems, and understanding the 
influence of laws and policies related to schools. 

A comprehensive review of the international literature on perceived problems of 
beginning teachers conducted by Veenman (1984) revealed the following problems: 
classroom discipline, motivating students, dealing with individual differences, assessing 
students' work, relations with parents, organization of class work, insufficient materials 
and supplies, dealing with problems of individual students, heavy teaching load 
resulting in insufficient preparation time, relations with colleagues, planning of lessons 
and school days, effective use of different teaching methods, awareness of school 
policies and rules, determining learning level of students, knowledge of subject matter, 
burden of clerical work, relations with principals/administrators, inadequate school 
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equipment, dealing with slow learners, dealing with students of different cultures and 
deprived backgrounds, effective use of textbooks and curriculum guides, lack of spare 
time, inadequate guidance and support, and large class size. 

Based upon the review of the literature, and relying heavily upon the work done 
by Veenman (1984), Sandidge developed an instrument that was administered to intern 
teachers and resource teachers during the initial implementation of the Kentucky 
Teacher Internship Program. The instrument included: knowledge of subject matter, 
presentation of subject matter, communication skills, classroom organization, time 
management, management of paperwork, daily lesson planning, long-term instructional 
planning, use of instructional materials, ability to meet individual needs, understanding 
of cultural differences, evaluation of student performance, management of student 
conduct, motivation of students, expectations for student work and behavior, 
relationships with parents, relationships with other teachers, relationships with school 
administrators, and knowledge of school and district policies and procedures. The 
intern and resource teachers were asked to identify the degree of difficulty that the 
intern teachers experienced in each area using a Likert scale ranging from one to five, 
from no difficulty to an extremely large degree of difficulty. 

Implementation of the Pilot Assessment 

As a result of Sandidge's findings, the College concluded that a follow-up 
instrument comprised of frequently perceived problems of first-year teachers completed 
by the graduate as well as by all of the other members of the Kentucky Teacher 
Internship Program team would provide valuable information to faculty for the 
purposes of program improvement. Consequently, with funding provided by the 
Appalachia Educational Laboratory for a pilot study, Sandidge's instrument was 
modified to identify twenty competencies. Two versions of the instrument were 
prepared. On one, fnst-year teachers were asked to rank the degree of difficulty they 
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experienced in each area. On the other, each member of the Kentucky Teacher 
Internship evaluation team was asked to rank the first-year teacher's proficiency on the 
same twenty items. Demographic data were also collected from each intern, and 
comments were ( sought from all participants in the follow-up study. Both versions of 
the revised instrument were evaluated by members of the Kentucky Association of 
Teacher Educators and the executive board of the Kentucky Association of Colleges for 
Teacher Education. 

Following the preparation of the instrument, the College worked with the 
Kentucky Department of Education to identify those interns who were graduates of the 
University of Kentucky, and a stratified sample was selected. Each intern in the 
sample was contacted and asked to participate and to sign a consent form allowing the 
other members of the Kentucky Teacher Internship team to participate as well. A small 
stipend ($10) was offered to interns and committee members upon completion of the 



assessment. 



Upon receipt of consent forms from the interns, the appropriate version of the 
instrument was disseminated to each member of the intern team. Team members were 
asked to complete the instrument independently of one another near the end of the 
internship year. Approximately 95 percent of the sample completed the study. At the 
completion of the pilot, the responses and comments made by the resource team 
members were analyzed. Also studied were those comments received relating to the 
instrument itself. It was determined that the process resulted in more specific 
information than had been previously received using the more traditional survey 
method. It was also determined that the method of eliciting feedback from all members 
of the graduate's internship team using a revised instrument should be continued. 
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Selected Findings and Results 

The University of Kentucky has continued to assess its teacher education 
graduates with the assistance of their internship team members. Each year, UK interns 
are identified and contacted to ascertain their willingness to evaluate themselves and to 
be evaluated by their other team members for the purpose of improving the teacher 
preparation program at the University (though stipends are no longer available). Data 
have now been collected for academic years 1988-89, 1989-90, 1990-91, and 1991-92. 
With the exception of 1991-92 when teachers were heavily involved in implementing 
component of the Kentucky Education Reform Act (KERA), approximately 70 percent 
of the graduates have responded to the survey each year and given permission for 
committee members to participate as well. 

The returned questionnaires produced rich and varied qualitative and 
quantitative data. Overall, both quantitative and qualitative responses have been 
complimentary of the University 1 s teacher education programs. Certain patterns have 
emerged from the data throughout the years. The table in Appendix I outlines 
summary data for a three-year period from 1989 to 1992 from graduates and the 
principals, resource teachers, and teacher educators who served on graduates' 
internship committees. Data presented in the table represent the percentage of 
graduates, principals, resource teachers, and teacher educators who rated the teaching 
performance of the graduates as proficient or higher (i.e., a rating of 3, 4, or 5) on the 
identified teacher competencies. 

The survey results contained in the table in Appendix i are based upon 
responses from 194 graduates, 119 principals, 122 resource teachers, and 101 teacher 
educators. Data indicate that a najority of graduates and their internship committee 
numbers believe that graduates are proficient in the 20 teacher competencies. As an 
example, on the first category, 100 percent of the graduates, principals, resource 
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teachers, and teacher educators rated graduates as proficient or highly proficient in the 
area of knowledge of subject matter. 

From the qualitative data can be found many statements of intern need for more 
coursework and practice in working with classroom management and discipline and for 
longer experiences in the schools prior to the internship year. At the same time, the 
respondents repeated their appreciation for their faculty, for their curriculum, and for 
the internship experience. From the quantitative data, two broad patterns emerged: (1) 
intern teachers appeared to be more critical of their abilities than were committee 
members and (2) a number of intern teachers and larger numbers of principals, teacher 
educators, and resource teachers reported they were unable to make judgments about 
the intern teachers 1 abilities to work with cultural differences and with individuals with 
handicapping conditions. 

Once data from each survey cycle are fully analyzed, feedback is provided to 
the various College program faculties who then use the findings to modify and improve 
programs. Through follow-up studies, College of Education graduates have expressed 
a need for additional assistance in various areas. Programs have responded by adding 
new courses, modifying existing courses, and offering professional development 
workshops, seminars, and conferences. Two areas that teachers have most frequently 
and consistently identified as problem areas have been classroom organization and 
management of student conduct. In addition to these requests for continuing 
professional development in these areas, a classroom management and discipline course 
has been instituted as an elementary program requirement. 

Graduates have also expressed a need for more extensive and earlier field 
experiences. As a result of this identified need, additional clinical and laboratory 
experiences have been added as teacher education requirements. According to survey 
responses, relationships with parents have created problems for some graduates; these 
teachers perceived themselves as lacking expertise in this area. Therefore, the course, 



Working with Parents, has become a requirement for early elementary education 
majors. 

With the availability of computer technology in school programs for disabled 
and gifted children, special education teachers have begun to request assistance with 
utilization of computers in the classroom. As a result of these requests and extramural 
funding in the area of special education technology, the Department of Special 
Education has offered graduate level programming and coursework designed to enable 
teachers to incorporate microcomputer technology into their instructional programs. In 
addition, the Department has instituted post-masters and post-doctoral programs in 
special education technology. 

Using input from graduates of the Vocational Agricultural Education Program, 
faculty members have expanded or strengthened several areas of the preparation 
program, including work with exceptional learners, providing guidance, using the 
computer, and record keeping. Also, at the request of graduates, graduate-level 
courses have been offered in off-campus locations convenient to students 1 residences 
and work sites. Feedback from vocational home economics graduates indicated a need 
for instruction in the areas of working with students with special needs and teaching 
entrepreneurship in home economics. Also, an additional course, Teaching Home 
Economics in the Middle School, was taught in an effort to assist those teachers 
teaching in junior high schools that were being converted to middle schools. Other 
courses taught in response to student feedback included: the use of computer 
technology in home economics instruction, teen pregnancy/parenthood education, and 
entrepreneurship in home economics. 

In response to requests from internship team member respondents in northern 
Kentucky, the Department of Administration and Supervision initiated joint 
programming with Northern Kentucky University in order to deliver a portion of its 

12 



Ed.D. program to northern Kentucky. This effort has now extended to western and 
eastern parts of the Commonwealth. 

Comments from graduates have also been considered in planning workshops, 
seminars, and conferences to meet the instructional and curricular needs of teachers. 
Professional development workshops ant seminars are primarily designed around the 
most frequently teacher-requested topics. 

Perhaps the most significant and comprehensive use of the data has been in the 
redesign of teacher education programs in the College. Charged with examining the 
content and delivery of the teacher education program, an interdisciplinary faculty 
committee relied heavily on the data provided by the graduates in assessing the 
effectiveness of the program. The committee noted that comments from graduates 
consistently suggested that: field experiences be increased, courses and instructors 
within the professional sequence be better linked, theory be directly related to practice, 
and the instruction of the political and administrative aspects of teaching be expanded. 
Consequently, when the faculty of the College adopted the committee's report in 1992, 
they responded to these comments by accepting recommendations for the development 
of a field-based, outcomes-based teacher education preparation program that integrated 
course instruction across disciplines. Teams of faculty with expertise in curriculum and 
instruction, administration, policy studies, special education, school law, multicultural 
education, technology, and content disciplines are working together to plan instruction 
for secondary education cohorts in an intensive, one-year, master's degree program. 
The instruction will be delivered in blocks of time-the professional sequence of 24 
hours of credit is contained in four courses-by varying combinations of team members. 
Continuous assessment of the students will be conducted by the instructional teams and 
by the program faculties responsible for making the student admissions a - j retention 
decisions. Students' knowledges and abilities will be assessed using portfolios and on- 
demand performance tasks twice during the program-once near the mid-point and once 




at the end. In addition to assuring that all students will demonstrate proficiency in 
meeting the outcomes specified by the program before graduation and licensure, this 
assessment process will provide the College with data that might be compared with the 
follow-up data to determine what, if any, proficiency diminishes in the time between 
completion of a program and securing a teaching position. 



Current Revisions to the Instrument 

It is clear that faculty in the College have found the information collected from 
the internship team members to be beneficial in their curriculum planning and wish f 
see the assessment continue. However, as the College became involved in the process 
of dramatically changing its teacher preparation program, requiring graduates to 
demonstrate proficiencies in areas additional to those currently assessed, it became 
apparent that a redesign of the follow-up instrument would be necessary. Redesign was 
also warranted given the changes that had already been made in the teacher education 
program to reflect implementation of the components of the Kentucky Education 
Reform Act. Consequently, during the 1992-93 academic year, a committee of faculty 
and administrators began to examine the instructional implications of the Kentucky 
Education Reform Act of 1990. The committee members also examined the teacher 
educator outcomes developed and adopted by the University of Kentucky faculty in 
1992 to determine whether proficiencies not already measured by the Colle^ s follow- 
up instrument should be added. 

The committee determined that some additional items should be assessed, 
developed those items, and revised the instrument to reflect the changes. This revised 
instrument (Appendix II) was field tested during the spring of *994, assessing 1993-94 
year interns. Data collected from the field test will be used to evaluate the revision of 
the instrument for future use in the assessment of teacher education program graduates 
at the University of Kentucky. 
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Summary 

The implementation of a teacher internship program in Kentucky presented an 
opportunity for the improvement of the follow-up evaluation of graduates being 
conducted by the College of Education at the University of Kentucky. An instrument 
identifying teacher competencies was developed and administered to the graduates and 
to the teacher educators, administrators, and resource teachers who comprised the 
graduates 1 intern support and evaluation teams. Data collected from these follow-up 
assessments have consistently shown that graduates of teacher preparation programs at 
the University of Kentucky are judged to be proficient by all members of the internship 
team. Even so, trend data and respondent comments have provided information that 
has allowed program faculties to continue to improve the teacher preparation program 
by creating new courses, revising existing courses, and developing in-service 
workshops and institutes. The value of the information gathered with the assistance of 
all members of the graduates 1 internship teams is viewed as so significant that the 
process is being adapted to reflect education reform and to assess the resulting 
expectations for teachers. 
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APPENDIX I 



Percentage of Graduates and Their Internship Committee Members Rating Graduates as 
Proficient or Above on Selected Teacher Competencies 

1989-92 



Teacher Competencies 


% of 
Graduates 
(n = 194) 


% of 
Principals 
(n = 119) 


% of 
Resource 
Teachers 
(n = 122) 


% of Teacher 1 
Educators 
(n^lOl) 


Knowledge of subject matter 


100 


100 


100 


100 


Presentation of subject matter 


100 


98 


100 


98 


Adapting instruction to cultural 
differences 


100 


98 


100 


97 


Adapting instruction to handicapping 
conditions 


94 


98 


100 


96 


Use of instructional media and supplies 


98 


99 


98 


98 


Evaluation of student performance 


100 


100 


100 


100 


Classroom organization 


98 


98 


98 


95 


Time management 


97 


98 


98 


97 


Management of paperwork 


96 


98 


99 


96 


Daily lesson planning 


97 


97 


98 


97 


Long-term instructional planning 


97 


98 


00 


An 

yy 


Communication skills 


100 


100 


98 


99 


Management of student conduct 


94 


90 


94 


97 


Motivation of students 


98 


97 


97 


100 


Expectations for student work and 
behavior 


99 


99 


98 


99 


Relationships with parents 


98 


97 


97 


98 


Relationships with other teachers 


99 


98 


96 


97 


Relationships with administrators 


100 


99 


97 


99 


Ability to meet individual needs 


99 


97 


98 


99 


Knowledge of school and district 
1 policies and procedures 


90 


97 


94 


97 



Scale: 1 (least proficient) to 3 (proficient) to 5 (highly proficient) 
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APPENDIX H 



FOLLOW-UP OF UNIVERSITY OF KENTUCKY GRADUATES 
PARTICIPATING IN THE KENTUCKY TEACHER INTERNSHIP PROGRAM 

Spring 1994 

The purpose of this follow-up study is to gather information regarding the teaching performance of intern teachers 
who are University of Kentucky graduates. This information will assist the College of Education in evaluating and 
improving its academic programs. In the remainder of this form are 24 competencies, five context descriptors, and 



demographic data. Please rate your own competencies on the 



and complete the remainder of the survey. Please circle your response. 



I . Knowledge of subject matter 
Presentation of subject matter 

3. Adapting instruction to cultural differences 

4. Adapting instruction to handicapping conditions 

5. Use of instructional media and supplies 

6. Evaluation of student performance 

7. Classroom organization 

8. Time management 

9. Management of paperwork 

10. Daily lesson planning 

I I. Long-term instructional planning 

12. Communication skills 

13. Management of student conduct 

14. Motivation of stude nts 

15. Expectations for student work and behavior 

16. Relationships with parents 

17. Relationships with other teachers 

18. Relationships with administrators 



least highly unable to 

proficient proficient proficient judge 



to 5 scale, rate your role within the school context, 



2 

2 

9 

2 
2 
2 
2 



2 
2 
2 
2 
2 
2 
2 



4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 



U 
U 
U 
U 
U 
U 
U 
U 
U 
U 

u 
u 
u 
u 
u 
u 
u 
u 
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19. Ability to meet individual needs 

20. Knowledge of school and district policies and 
procedures 

21 . Integration of content across curriculum 

22. Knowledgf of portfolio development 

23. Development of authentic performance tasks 

24. Reflection and evaluation of teaching and learning 



2 
2 
2 
2 



4 
4 

4 
4 
4 
4 



U 
U 

u 
u 
u 
u 



25. 



26. 



28. 
29. 
30. 
31. 
32. 
33. 
34. 
35. 



The educational context of the classroom and school in which I teach is (please circle the number of 
the more appropriate response): 

1 an unusually demanding classroom and/or school in which to work 

2 more demanding than most teaching situations in schools of this size and location 

3 about like most schools of this size and location 

4 less demanding than most teaching situations in schools of this size and location 

5 an easy classroom and school in which to teach 

Race/ethnicity: 

1 African American 

2 Asian American 

3 Hispanic 

4 White 

5 Other 

Gender: 

1 Male 

2 Female 

Age (please indicate in years): 



Grade level(s) currently teaching: 
Subject area(s) currently teaching: 



Semester and year graduated or completed program: 
Degree and/or certification program completed: 
Academic program/major:__ 



Grade level(s) taught during student teaching: 
Subject area(s) taught during student teaching: 
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36. What were the strengths of your preparation program at the College of Education? 



37. What recommendations or suggestions do you have for improving the program in which you were 
enrolled? 
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